Autonomous correction of alignment error in a master-slave robotic system

ABSTRACT

In some embodiments, correcting an alignment error between an end effector of a tool associated with a slave and a master actuator associated with a master in a robotic system involves receiving at the master, master actuator orientation signals (RMCURR) representing the orientation of the master actuator relative to a master reference frame and generating end effector orientation signals (REENEW) representing the end effector orientation relative to a slave reference frame, producing control signals based on the end effector orientation signals, receiving an enablement signal for selectively enabling the control signals to be transmitted from the master to the slave, responsive to a transition of the enablement signal from not active state to active state, computing the master-slave misalignment signals (RΔ) as a difference between the master actuator orientation signals (RMCURR) and the end effector orientation signals (REENEW), and adjusting the master-slave misalignment signals (RΔ) to reduce the alignment difference.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/411,537, filed on May 14, 2019, and issued as U.S. Pat. No.10,820,953, which is a continuation of U.S. patent application Ser. No.15/542,356, filed on Jul. 7, 2017, and issued as U.S. Pat. No.10,327,856, which is a U.S. national stage of International ApplicationNo. PCT/CA2016/000006, filed on Jan. 8, 2016, which claims priority toU.S. Provisional Application No. 62/101,731, filed on Jan. 9, 2015. Eachof these applications is incorporated by reference in its entirety.

BACKGROUND OF THE INVENTION 1. Field of Invention

This invention relates to master-slave robotic systems such as used forlaparoscopic surgery and more particularly to autonomous correction ofalignment error between the master and slave in such systems.

2. Description of Related Art

During operation of a teleoperated robotic minimally invasive surgicalsystem where a slave instrument is intended to follow the motion of amaster input controller held by a user, it is possible for the masterand slave to become misaligned with respect to each other's base framesof reference. In instances of misalignment, the slave end-effector nolonger points in the direction that the user is expecting, which mayresult in less than optimal controllability of the slave instrument.

U.S. Pat. No. 7,806,891 entitled “Repositioning and reorientation ofmaster slave relationship in minimally invasive telesurgery” describes asystem but the system disrupts motion of the slave when there ismisalignment between the master and slave. The system also requires amaster controller with the ability to actively control, or lockorientation degrees of freedom.

U.S. Pat. No. 8,423,186 entitled “Ratcheting for master alignment of ateleoperated minimally invasive surgical instrument” describes a systemthat only reduces alignment error when the motion of the master handleis in a direction such that it would reduce the misalignment if theslave were not moved.

SUMMARY OF THE INVENTION

The disclosure describes a method of correcting an alignment errorbetween an end effector of a tool associated with a slave and a masteractuator associated with a master in a master-slave robotic system inwhich an orientation of the end effector is remotely controlled by anorientation of the master actuator by producing and transmitting controlsignals at the master for controlling the slave. The method involvescausing a processor associated with the master to receive masteractuator orientation signals (R_(MCURR)) representing the orientation ofthe master actuator relative to a master reference frame and causing theprocessor to generate end effector orientation signals (R_(EENEW))representing the end effector orientation relative to a slave referenceframe, in response to the master actuator orientation signals (R_(m)),master base orientation signals (R_(MBASE)) representing previous-savedvalues of the master actuator orientation signals (R_(MCURR)) and slavebase orientation signals (R_(EEBASE)) representing previous-saved valuesof the end effector orientation signals (R_(EENEW)). The method alsoinvolves causing the processor to produce control signals based on theend effector orientation signals, for transmission from the master tothe slave and causing the processor to receive an enablement signal forselectively enabling the control signals to be transmitted from themaster to the slave whereby the master transmits the control signals tothe slave when the enablement signal is active and does not transmit thecontrol signals to the slave when the enablement signal is not activeand such that when the enablement signal is active, changes in theorientation of the master actuator cause corresponding changes in theorientation of the end effector and such that when the enablement signalis not active, changes in the orientation of the master actuator do notcause corresponding changes in the orientation of the end effector. Themethod also involves, when the enablement signal transitions from thenot active state to the active state, causing the processor to save thevalues of the master actuator orientation signals (R_(MCURR)) as themaster base orientation signals (R_(MBASE)) to create the previous-savedvalues of the master actuator orientation signals (R_(MCURR)) and savethe values of the end effector orientation signals (R_(EENEW)) as theslave base orientation signals (R_(EEBASE)) to create the previous-savedvalues of the end effector orientation signals (R_(EENEW)). The methodalso involves causing the processor to detect a difference, between themaster actuator orientation signals (R_(MCURR)) and the end effectororientation signals (R_(EENEW)), the difference representing adifference in physical alignment between the tool and the masterrelative to their respective reference frames and in response todetecting the difference, causing the processor to adjust the savedslave base orientation signals (R_(EEBASE)) to ultimately have the samevalues as the saved master base orientation (R_(MBASE)) values so thatsubsequent generations of the end effector orientation signals(R_(EENEW)) cause the control signals to cause the tool to satisfy analignment criterion.

The disclosure also describes a method of correcting an alignment errorbetween an end effector of a tool associated with a slave and a masteractuator associated with a master in a master-slave robotic system inwhich an orientation of the end effector is remotely controlled by anorientation of the master actuator by producing and transmitting controlsignals at the master for controlling the slave. The method involvescausing a processor associated with the master to receive masteractuator orientation signals (R_(MCURR)) representing the orientation ofthe master actuator relative to a master reference frame and causing theprocessor to generate end effector orientation signals (R_(EENEW))representing the end effector orientation relative to a slave referenceframe, in response to the master actuator orientation signals(R_(MCURR)) and master-slave misalignment signals (R_(Δ)), representinga product of previously saved values of the master actuator orientationsignals (R_(MCURR)) and the end effector orientation signals(R_(EENEW)). The method also involves causing the processor to producecontrol signals based on the end effector orientation signals, fortransmission from the master to the slave and causing the processor toreceive an enablement signal for selectively enabling the controlsignals to be transmitted from the master to the slave whereby themaster transmits the control signals to the slave when the enablementsignal is active and does not transmit the control signals to the slavewhen the enablement signal is not active and such that when theenablement signal is active, changes in the orientation of the masteractuator cause corresponding changes in the orientation of the endeffector and such that when the enablement signal is not active, changesin the orientation of the master actuator do not cause correspondingchanges in the orientation of the end effector. The method alsoinvolves, when the enablement signal transitions from the not activestate to the active state, causing the processor to compute themaster-slave misalignment signals (R_(Δ)) as a difference between themaster actuator orientation signals (R_(MCURR)) and the end effectororientation signals (R_(EENEW)), the misalignment signals representing adifference in physical alignment between the tool and the masterrelative to their respective reference frames. The method furtherinvolves causing the processor to detect a second difference between themaster actuator orientation signals (R_(MCURR)) and the end effectororientation signals (R_(EENEW)) the second difference representing adifference in physical alignment between the tool and the masterrelative to their respective reference frames and in response todetecting the second difference, causing the processor to adjust themaster-slave misalignment signals (R_(Δ)) to reduce the alignmentdifference to satisfy an alignment criterion so that subsequentgenerations of the end effector orientation signals (R_(EENEW)) causethe control signals to cause the tool to be physically aligned with themaster within the alignment criterion.

The disclosure also describes an apparatus for correcting an alignmenterror between an end effector of a tool associated with a slave and amaster actuator associated with a master in a master-slave roboticsystem in which an orientation of the end effector is remotelycontrolled by an orientation of the master actuator by producing andtransmitting control signals at the master for controlling the slave.The apparatus includes means associated with the master for receivingmaster actuator orientation signals (R_(MCURR)) representing theorientation of the master actuator relative to a master reference frameand means for generating end effector orientation signals (R_(EENEW))representing the end effector orientation relative to a slave referenceframe, in response to the master actuator orientation signals (R_(m)),master base orientation signals (R_(MBASE)) representing previous-savedvalues of the master actuator orientation signals (R_(MCURR)) and slavebase orientation signals (R_(EEBASE)) representing previous-saved valuesof the end effector orientation signals (R_(EENEW)). The apparatus alsoincludes means for producing control signals based on the end effectororientation signals, for transmission from the master to the slave andmeans for receiving an enablement signal for selectively enabling thecontrol signals to be transmitted from the master to the slave wherebythe master transmits the control signals to the slave when theenablement signal is active and does not transmit the control signals tothe slave when the enablement signal is not active and such that whenthe enablement signal is active, changes in the orientation of themaster actuator cause corresponding changes in the orientation of theend effector and such that when the enablement signal is not active,changes in the orientation of the master actuator do not causecorresponding changes in the orientation of the end effector. Theapparatus also includes means responsive to a transition of theenablement signal from the not active state to the active state, forsaving the values of the master actuator orientation signals (R_(MCURR))as the master base orientation signals (R_(MBASE)) to create theprevious-saved values of the master actuator orientation signals(R_(MCURR)) and saving the values of the end effector orientationsignals (R_(EENEW)) as the slave base orientation signals (R_(EEBASE))to create the previous-saved values of the end effector orientationsignals (R_(EENEW)). The apparatus also includes means for detecting adifference, between the master actuator orientation signals (R_(MCURR))and the end effector orientation signals (R_(EENEW)), the differencerepresenting a difference in physical alignment between the tool and themaster relative to their respective reference frames means for causingthe processor to adjust the saved slave base orientation signals(R_(EEBASE)) to ultimately have the same values as the saved master baseorientation (R_(MBASE)) values so that subsequent generations of the endeffector orientation signals (R_(EENEW)) cause the control signals tocause the tool to satisfy an alignment criterion, in response todetecting the difference.

The disclosure also describes an apparatus for correcting an alignmenterror between an end effector of a tool associated with a slave and amaster actuator associated with a master in a master-slave roboticsystem in which an orientation of the end effector is remotelycontrolled by an orientation of the master actuator by producing andtransmitting control signals at the master for controlling the slave.The apparatus includes means associated with the master for receivingmaster actuator orientation signals (R_(MCURR)) representing theorientation of the master actuator relative to a master reference frameand means for generating end effector orientation signals (R_(EENEW))representing the end effector orientation relative to a slave referenceframe, in response to the master actuator orientation signals(R_(MCURR)) and master-slave misalignment signals (R_(Δ)), representinga product of previously saved values of the master actuator orientationsignals (R_(MCURR)) and the end effector orientation signals(R_(EENEW)). The apparatus also includes means for producing controlsignals based on the end effector orientation signals, for transmissionfrom the master to the slave and means for receiving an enablementsignal for selectively enabling the control signals to be transmittedfrom the master to the slave whereby the master transmits the controlsignals to the slave when the enablement signal is active and does nottransmit the control signals to the slave when the enablement signal isnot active and such that when the enablement signal is active, changesin the orientation of the master actuator cause corresponding changes inthe orientation of the end effector and such that when the enablementsignal is not active, changes in the orientation of the master actuatordo not cause corresponding changes in the orientation of the endeffector. The apparatus also includes means responsive to a transitionof the enablement signal from the not active state to the active state,for computing the master-slave misalignment signals (R_(Δ)) as adifference between the master actuator orientation signals (R_(MCURR))and the end effector orientation signals (R_(EENEW)), the misalignmentsignals representing a difference in physical alignment between the tooland the master relative to their respective reference frames, means fordetecting a second difference, between the master actuator orientationsignals (R_(MCURR)) and the end effector orientation signals (R_(EENEW))the second difference representing the difference in physical alignmentbetween the tool and the master relative to their respective referenceframes and means responsive to detecting the second difference, forcausing the processor to adjust the master-slave misalignment signals(R_(Δ)) to reduce the alignment difference to satisfy an alignmentcriterion so that subsequent generations of the end effector orientationsignals (R_(EENEW)) cause the control signals to cause the tool to bephysically aligned with the master within the alignment criterion.

The disclosure also describes an apparatus for correcting an alignmenterror between an end effector of a tool associated with a slave and amaster actuator associated with a master in a master-slave roboticsystem in which an orientation of the end effector is remotelycontrolled by an orientation of the master actuator by producing andtransmitting control signals at the master for controlling the slave.The apparatus includes a processor associated with the master operablyconfigured to receive master actuator orientation signals (R_(MCURR))representing the orientation of the master actuator relative to a masterreference frame. The processor is further configured to generate endeffector orientation signals (R_(EENEW)) representing the end effectororientation relative to a slave reference frame, in response to themaster actuator orientation signals (R_(m)), to generate master baseorientation signals (R_(MBASE)) representing previous-saved values ofthe master actuator orientation signals (R_(MCURR)) and to generateslave base orientation signals (R_(EEBASE)) representing previous-savedvalues of the end effector orientation signals (R_(EENEW)). The processis also configured to produce control signals based on the end effectororientation signals, for transmission from the master to the slave andto receive an enablement signal for selectively enabling the controlsignals to be transmitted from the master to the slave whereby themaster transmits the control signals to the slave when the enablementsignal is active and does not transmit the control signals to the slavewhen the enablement signal is not active and such that when theenablement signal is active, changes in the orientation of the masteractuator cause corresponding changes in the orientation of the endeffector and such that when the enablement signal is not active, changesin the orientation of the master actuator do not cause correspondingchanges in the orientation of the end effector. The processor is alsoconfigured to, when the enablement signal transitions from the notactive state to the active state, save the values of the master actuatororientation signals (R_(MCURR)) as the master base orientation signals(R_(MBASE)) to create the previous-saved values of the master actuatororientation signals (R_(MCURR)) and save the values of the end effectororientation signals (R_(EENEW)) as the slave base orientation signals(R_(EEBASE)) to create the previous-saved values of the end effectororientation signals (R_(EENEW)). The processor is also configured todetect a difference, between the master actuator orientation signals(R_(MCURR)) and the end effector orientation signals (R_(EENEW)), thedifference representing a difference in physical alignment between thetool and the master relative to their respective reference frames andthe processor is configured to adjust the saved slave base orientationsignals (R_(EEBASE)) to ultimately have the same values as the savedmaster base orientation (R_(MBASE)) values so that subsequentgenerations of the end effector orientation signals (R_(EENEW)) causethe control signals to cause the tool to satisfy an alignment criterion,in response to detecting the difference.

The disclosure also describes an apparatus for correcting an alignmenterror between an end effector of a tool associated with a slave and amaster actuator associated with a master in a master-slave roboticsystem in which an orientation of the end effector is remotelycontrolled by an orientation of the master actuator by producing andtransmitting control signals at the master for controlling the slave.The apparatus includes a processor associated with the master andoperably configured to receive master actuator orientation signals(R_(MCURR)) representing the orientation of the master actuator relativeto a master reference frame. The processor is further configured togenerate end effector orientation signals (R_(EENEW)) representing theend effector orientation relative to a slave reference frame, inresponse to the master actuator orientation signals (R_(MCURR)) andmaster-slave misalignment signals (R_(Δ)), representing a product ofpreviously saved values of the master actuator orientation signals(R_(MCURR)) and the end effector orientation signals (R_(EENEW)). Theprocessor is also configured to produce control signals based on the endeffector orientation signals, for transmission from the master to theslave and receive an enablement signal for selectively enabling thecontrol signals to be transmitted from the master to the slave wherebythe master transmits the control signals to the slave when theenablement signal is active and does not transmit the control signals tothe slave when the enablement signal is not active and such that whenthe enablement signal is active, changes in the orientation of themaster actuator cause corresponding changes in the orientation of theend effector and such that when the enablement signal is not active,changes in the orientation of the master actuator do not causecorresponding changes in the orientation of the end effector. Theprocessor is also configured to, in response to a transition of theenablement signal from the not active state to the active state, computethe master-slave misalignment signals (R_(Δ)) as a difference betweenthe master actuator orientation signals (R_(MCURR)) and the end effectororientation signals (R_(EENEW)), the misalignment signals representing adifference in physical alignment between the tool and the masterrelative to their respective reference frames. The processor is alsoconfigured to detect a second difference, between the master actuatororientation signals (R_(MCURR)) and the end effector orientation signals(R_(EENEW)) the second difference representing the difference inphysical alignment between the tool and the master relative to theirrespective reference frames and responsive to detecting the seconddifference, the processor adjusts the master-slave misalignment signals(R_(Δ)) to reduce the alignment difference to satisfy an alignmentcriterion so that subsequent generations of the end effector orientationsignals (R_(EENEW)) cause the control signals to cause the tool to bephysically aligned with the master within the alignment criterion.

Other aspects and features of the present invention will become apparentto those ordinarily skilled in the art upon review of the followingdescription of specific embodiments of the invention in conjunction withthe accompanying figures.

BRIEF DESCRIPTION OF THE DRAWINGS

In drawings which illustrate embodiments of the invention,

FIG. 1 is a pictorial representation of a laparoscopic surgery systemaccording to one embodiment of the invention;

FIG. 2 is an oblique view of an input device of a master subsystem ofthe laparoscopic surgery system shown in FIG. 1;

FIG. 3 is a block diagram illustrating certain functionality and certainsignals produced and used by the system shown in FIG. 1;

FIG. 4 is an oblique view of a tool positioning device with a tool inthe form of an end effector held thereby, in an insertion tube of thelaparoscopic surgery system shown in FIG. 1;

FIG. 5 is a schematic representation of current and previous valuebuffers maintained by a master apparatus of the laparoscopic surgerysystem shown in FIG. 1 and updated according to the functions shown inFIGS. 6, 8A and 8B;

FIG. 6 is a block diagram of a storage routine executed by the masterapparatus in response to detection of a signal transition of anenablement signal produced in response to user input;

FIG. 7 is an oblique view of the input device shown in FIG. 2 and thetool positioning device shown in FIG. 4 showing relationships betweenbase axes of the input device and the end effector;

FIGS. 8A-8E are successive portions of a flowchart representing codesexecuted by the master apparatus of the laparoscopic surgery systemshown in FIG. 1, to provide for computation of an alignment differencebetween the input device shown in FIG. 2 and the end effector shown inFIG. 4 and for controlling translational movement of the end effectorand for controlling the type of control signals sent to a slavesubsystem of the laparoscopic surgery system shown in FIG. 1, based onthe computed alignment difference; and

FIG. 9 is a flowchart of an alternative block of code optionallyreplacing the block of code shown at 204 and 206 in FIG. 8B.

DETAILED DESCRIPTION

Referring to FIG. 1, a robotic control system in the form of alaparoscopic surgery system is shown generally at 50. The systemincludes a master subsystem 52 and a slave subsystem 54. The mastersubsystem 52 may be located anywhere in the world, but for the purposesof this description it will be considered to be in an operating room.The slave subsystem 54 is located in the operating room.

In the embodiment shown, the master subsystem 52 comprises a workstation56 having first and second input devices 58 and 60 and a viewer 62 incommunication with a master apparatus 64 comprising at least oneprocessor. The first and second input devices 58 and 60 are operable tobe actuated by respective hands of an operator such as a surgeon, forexample, who will perform the laparoscopic surgery by manipulating thefirst and second input devices of the master subsystem 52 to controlcorresponding laparoscopic tools 66 and 67 on the slave subsystem 54.

The viewer 62 may include an LCD display 68, for example, for displayingimages acquired by a camera 70 on the slave subsystem 54, to enable theuser to see the laparoscopic tools 66 and 67 inside the patient whilemanipulating the first and second input devices 58 and 60 to cause thetools to move in desired ways to perform the surgery. The first andsecond input devices 58 and 60 produce position and orientation signalsthat are received by the master apparatus 64 and the master apparatusproduces slave control signals that are transmitted by wires 72 orwirelessly, for example, from the master subsystem 52 to the slavesubsystem 54.

The slave subsystem 54 includes a slave computer 74 that receives theslave control signals from the master subsystem 52 and produces motorcontrol signals that control motors 76 on a drive mechanism of a toolcontroller 78 of the slave subsystem 54, to extend and retract wires(not shown) of respective tool positioning devices 79 and 81 to positionand to rotate the tools 66 and 67. Exemplary tool positioning devicesand tools for this purpose are described in PCT/CA2013/001076, which isincorporated herein by reference. The tool positing devices 79 and 81extend through an insertion tube 61, a portion of which is insertedthrough a small opening 63 in the patient to position end effectors 71and 73 of the tools 66 and 67 inside the patient, to facilitate thesurgery.

In the embodiment shown, the workstation 56 has a support 80 having aflat surface 82 for supporting the first and second input devices 58 and60 in positions that are comfortable to the operator whose hands areactuating the first and second input devices 58 and 60.

In the embodiment shown, the slave subsystem 54 includes a cart 84 inwhich the slave computer 74 is located. The cart 84 has an articulatedarm 86 mechanically connected thereto, with a tool holder mount 88disposed at a distal end of the articulated arm.

In the embodiment shown, the first and second input devices 58 and 60are the same, but individually adapted for left and right hand userespectively. In this embodiment, each input device 58 and 60 is anOmega.7 haptic device available from Force Dimension, of Switzerland.For simplicity, only input device 60 will be described, it is beingunderstood that input device 58 operates in the same way.

Referring to FIG. 2, generally input device 60 includes a base plate 90that supports a control unit 92 having arms 94, 96, 98 connected to agimbal-mounted handle 102 that can be grasped by the hand of theoperator user and rotated about orthogonal axes x₁, y₁ and z₁ of a firstCartesian reference frame having an origin at a point midway along theaxis of a cylinder that forms part of the handle 102. This firstCartesian reference frame may be referred to as the handle referenceframe. The origin may be referred to as the handle position 104.

The arms 94, 96, 98 facilitate translational movement of the handle 102and hence the handle position 104, in space, and confine the movement ofthe handle position within a volume in space. This volume may bereferred to as the handle translational workspace.

The handle 102 is mounted on a gimbal mount 106 having a pin 108. Thebase plate 90 has a calibration opening 110 for receiving the pin 108.When the pin 108 is received in the opening 110, the input device 60 isin a calibration position that is defined relative to a fixed masterCartesian reference frame comprising orthogonal axes x_(r), y_(r), z_(r)generally in the center of the handle translational workspace. In theembodiment shown, this master reference frame has an x_(r)-z_(r) planeparallel to the flat surface 82 and a y_(r) axis perpendicular to theflat surface.

In the embodiment shown, the z_(r) axis is parallel to the flat surface82 and is coincident with an axis 112 passing centrally through thecontrol unit 92 so that pushing and pulling the handle 102 toward andaway from the center of the control unit 92 along the axis 112 in adirection parallel to the flat surface 82 is movement in the z_(r)direction.

The control unit 92 has sensors (not shown) that sense the positions ofthe arms 94, 96, 98 and the rotation of the handle 102 and producessignals representing the handle position 104 (i.e. the center of thehandle 102) in the workspace and the rotational orientation of thehandle 102 relative to the fixed master reference frame x_(r), y_(r),z_(r). In this embodiment, these position and orientation signals aretransmitted on wires 111 of a USB bus to the master apparatus 64. Moreparticularly, the control unit 92 produces current handle positionsignals and current handle orientation signals that represent thecurrent position and orientation of the handle 102 by a current handleposition vector

_(MCURR) and a current handle rotation matrix R_(MCURR), relative to thefixed master reference frame x_(r), y_(r), z_(r).

For example, the current handle position vector

_(MCURR) is a vector

$\begin{Bmatrix}x_{1} \\y_{1} \\z_{1}\end{Bmatrix},$where x₁, y₁, and z₁ represent coordinates of the handle position 104within the handle workspace relative to the fixed master referenceframe, x_(r), y_(r), z_(r).

The current handle rotation matrix R_(MCURR) is a 3×3 matrix

$\begin{bmatrix}x_{1x} & y_{1x} & z_{1x} \\x_{1y} & y_{1y} & z_{1y} \\x_{1z} & y_{1z} & z_{1z}\end{bmatrix},$where the columns of the matrix represent the axes of the handlereference frame x₁, y₁, z₁ written in the fixed master reference framex_(r), y_(r), z_(r). R_(MCURR) thus defines the current rotationalorientation of the handle 102 in the handle translational workspace,relative to the x_(r), y_(r), z_(r) master reference frame.

The current handle position vector

_(MCURR) and current handle rotation matrix R_(MCURR) are transmitted inthe current handle position and orientation signals on wires 111 of theUSB bus, for example, to the master apparatus 64 in FIG. 1.

In addition, in the embodiment shown, referring to FIG. 1, the masterapparatus 64 is coupled to a footswitch 170 actuable by the operator(surgeon) to provide a binary enablement signal to the master apparatus64. When the footswitch 170 is not activated, i.e. not depressed, theenablement signal is in an active state and when the footswitch 170 isdepressed the enablement signal is in an inactive state. The footswitch170 thus controls the state of the enablement signal. As will be seenbelow, the enablement signal allows the user to cause the masterapparatus 64 to selectively enable and disable movement of the endeffectors in response to movement of the handles 102.

Referring now to FIG. 7, the end effector 73 and related structures aredescribed. The fixed slave reference frame has axes x_(s), y_(s) andz_(s) which intersect at a point referred to as the slave fixed baseposition 128, lying on the longitudinal axis 136 of the insertion tube61 and contained in a plane perpendicular to the longitudinal axis 136and containing a distal edge 103 of the insertion tube 61. The z_(s)axis is coincident with the longitudinal axis 136 of the insertion tube61. The x_(s)-z_(s) plane thus contains the longitudinal axis 136 of theinsertion tube 61 and the x_(s) and y_(s) axes define a planeperpendicular to the longitudinal axis 136 of the insertion tube 61.

In the embodiment shown, end effector 73 includes a pair of gripperjaws. Orthogonal axes x₂, y₂ and z₂ of an end effector Cartesianreference frame have an origin on the end effector axis, for example, atthe intersection at the tip of the gripper jaws of the end effector 73.The origin of the end effector reference frame may be referred to as theend effector position 150 relative to the fixed slave reference framex_(s), y_(s), z_(s). Due to the mobility of the tool positioning device81 and the mobility of the end effector 73 itself, the end effectorposition 150 can be placed at discrete positions within a volume inspace. This volume may be referred to as the end effector translationalworkspace.

New end effector positions and end effector orientations are calculatedby an end effector position and orientation calculation block 116encoded in the master apparatus 64 shown in FIG. 3, in response to thecurrent handle position signals

_(MCURR) and current handle orientation signals R_(MCURR) and arerepresented by a new end effector position vector

_(EENEW) and a new end effector rotation matrix R_(EENEW), relative tothe x_(s), y_(s), z_(s) fixed slave reference frame.

For example, the new end effector position vector

_(EENEW) is a vector

$\begin{Bmatrix}x_{2} \\y_{2} \\z_{2}\end{Bmatrix},$where x₂, y₂, and z₂ represent coordinates of the end effector position150 within the end effector translational workspace relative to thex_(s), y_(s), z_(s) fixed slave reference frame.

The end effector rotation matrix R_(EENEW) is a 3×3 matrix

$\begin{bmatrix}x_{2x} & y_{2x} & z_{2x} \\x_{2y} & y_{2y} & z_{2y} \\x_{2z} & y_{2z} & z_{2z}\end{bmatrix},$where the columns of the R_(EENEW) matrix represent the axes of the endeffector reference frame x₂, y₂, z₂ written in the fixed slave referenceframe x_(s), y_(s), z_(s). R_(EENEW) thus defines a new orientation ofthe end effector 73 in the end effector translational workspace,relative to the x_(s), y_(s), z_(s) reference frame.

Referring back to FIG. 1, in the embodiment shown, the master apparatus64 is controlled by program codes stored on a non-transitory computerreadable medium such as a disk drive 114. The codes direct the masterapparatus 64 to perform various functions. Referring to FIGS. 1 and 3,these functions may be grouped into categories and expressed asfunctional blocks of code including an end effector position andorientation calculation block 116, a kinematics block 118, a motioncontrol block 120, and a base setting block 216, all stored on the diskdrive 114 of the master apparatus 64. For ease of description, theseblocks are shown as functional blocks within the master apparatus 64 inFIG. 3. These functional blocks are executed separately but in the samemanner for each input device 58 and 60. The execution of thesefunctional blocks for only input device 60 and end effector 73 will bedescribed, it being understood they are separately executed in the sameway for input device 58 and end effector 71 to achieve control of endeffectors 73 and 71 by right and left hands respectively of theoperator.

Generally, the end effector position and orientation calculation block116 includes codes that direct the master apparatus 64 to produce thenew end effector position and orientation signals, referred to herein as{right arrow over (P)}_(EENEW) and R_(EENEW) n respectively.

The kinematics block 118 includes codes that direct the master apparatus64 to produce configuration variables in response to the newlycalculated end effector position and orientation signals {right arrowover (P)}_(EENEW) and R_(EENEW).

The motion control block 120 includes codes that direct the masterapparatus 64 to produce the slave control signals, in response to theconfiguration variables.

The base setting block 216 is executed asynchronously, whenever theenablement signal transitions from an inactive state to an active state,such as when the user releases the footswitch 170. The base settingblock 216 directs the master apparatus 64 to set new reference positionsand orientations for the handle 102 and end effector 73, respectively aswill be described below.

Referring back to FIG. 1, in the embodiment shown, the slave controlsignals represent wire length values indicating how much certain wiresof a given tool positioning device 81 of the slave subsystem 54 must beextended or retracted to cause the end effector 73 of the tool 67 to bepositioned and rotated in a manner determined by positioning androtating the corresponding input device 60.

Referring to FIGS. 1 and 3, the slave control signals representing thewire length values are transmitted to the slave computer 74, which hasits own computer readable medium encoded with a communication interfaceblock 124 including codes for directing the slave computer to receivethe slave control signals from the master apparatus 64. The computerreadable medium is also encoded with a motor control signal generatorblock 126 including codes for causing the slave computer 74 to generatemotor control signals for controlling the motors 76 on the toolcontroller 78 to extend and retract the wires controlling the attachedtool positioning device 81 according to the wire length valuesrepresented by the slave control signals from the master apparatus 64.

The kinematics block 118 receives newly calculated end effector positionand orientation signals {right arrow over (P)}_(EENEW) and R_(EENEW)each time the end effector position and orientation calculation block116 is executed. In response, the kinematics block 118 produces theconfiguration variables described below.

Referring to FIGS. 3 and 4, generally, the codes in the kinematics block118 direct the master apparatus 64 to calculate values for the aboveconfiguration variables in response to the end effector position androtation signals {right arrow over (P)}_(EENEW) and R_(EENEW) producedby the end effector position and orientation calculation block 116 andthese calculated configuration values generally define a toolpositioning device pose required to position end effector 73 at adesired location and at a desired orientation in the end effectortranslational workspace.

Referring to FIG. 4, the tool positioning device 81 has a firstarticulated segment 130, referred to as an s-segment and a secondarticulated segment 132 referred to as a distal segment. The segments130 and 132 each include a plurality of “vertebra” 324. The s-segment130 begins at a distance from the insertion tube 61, referred to as theinsertion distance q_(ins), which is the distance between the fixedslave base position 128 defined as the origin of the slave fixed basereference frame x_(s), y_(s), z_(s) and a first position 330 at theorigin of a first position reference frame x₃, y₃, and z₃. The insertiondistance q_(ins) represents an unbendable portion of the toolpositioning device 81 that extends out of the end of the insertion tube61. In the embodiment shown, the insertion distance q_(ins) may be about10-20 mm, for example. In other embodiments, the insertion distanceq_(ins) may be longer or shorter, varying from 0-100 mm, for example.

The s-segment 130 extends from the first position 330 to a thirdposition 334 defined as an origin of a third reference frame having axesx₅, y₅, and z₅ and is capable of assuming a smooth S-shape when controlwires (not shown) inside the s-segment 130 are pushed and pulled. Thes-segment 130 has a mid-point at a second position 332, defined as theorigin of a second position reference frame having axes x₄, y₄, z₄. Thes-segment 130 has a length L₁, which in the embodiment shown may beabout 65 mm, for example.

The distal segment 132 extends from the third position 334 to a fourthposition 336 defined as an origin of a fourth reference frame havingaxes x₆, y₆, z₆. The distal segment 132 has a length L₂, which in theembodiment shown may be about 23 mm, for example.

The tool 67 also has an end effector length, which in the embodimentshown is a gripper length L₃ that extends from the fourth position 336to the end effector position 150 defined as the origin of axes x₂, y₂,and z₂. The gripper length L₃, in this embodiment, may be about 25 mm,for example. The slave base position 128, first position 330, secondposition 332, third position 334, fourth position 336 and end effectorposition 150 may collectively be referred to as tool referencepositions.

As explained in PCT/CA2013/001076, hereby incorporated herein byreference in its entirety, by pushing and pulling on certain controlwires inside the tool positioning devices 79 and 81, the s-segment 130can be bent into any of various degrees of an S-shape, from straight asshown in FIG. 7 to a partial S-shape as shown in FIG. 4 to a fullS-shape. The s-segment 130 is sectional in that it has a first section320 and a second section 322 on opposite sides of the second position332. The first and second sections 320 and 322 lie in a first bend planecontaining the first position 330, second position 332, and thirdposition 334. The first bend plane is at an angle δ_(prox) to thex_(s)-z_(s) plane of the fixed slave reference frame. The first section320 and second section 322 are bent in the first bend plane throughopposite but equal angles θ_(prox) such that no matter the angleθ_(prox) or the bend plane angle δ_(prox), the z_(s) axis of the thirdposition 334 is always parallel to and aligned in the same direction asthe z_(s) axis of the fixed slave base position 128. Thus, by pushingand pulling on the control wires within the tool positioning device 81,the third position 334 can be placed at any of a number of discretepositions within a cylindrical volume in space. This volume may bereferred to as the s-segment workspace.

In addition, the distal segment 132 lies in a second bend planecontaining the third position 334 and the fourth position 336. Thesecond bend plane is at an angle δ_(dist) to the x_(s)-z_(s) plane ofthe fixed slave reference frame. The distal segment 132 is bent in thesecond bend plane at an angle θ_(dist). Thus, by pushing and pulling thecontrol wires within the tool positioning device 81, the fourth position336 can be placed within another volume in space. This volume may bereferred to as the distal workspace. The combination of the s-segmentworkspace plus the distal workspace can be referred to as the toolpositioning device workspace, as this represents the total possiblemovement of the tools 66 and 67 as effected by the respective toolpositioning devices 79 and 81.

The distance between the fourth position 336 and the end effectorposition 150 is the distance between the movable portion of the distalsegment 132 and the tip of the gripper end effector 73 in the embodimentshown, i.e. the length L₃. Generally, the portion of the gripper betweenthe fourth position 336 and the end effector position 150 (L₃) will beunbendable.

In the embodiment shown, the end effector 73 is a gripper jaw tool thatis rotatable about the z₂ axis in the x₂-y₂ plane of the end effectorreference frame, the angle of rotation being represented by an angle γrelative to the positive x₂ axis. Finally, the gripper jaws may be atany of varying degrees of openness from fully closed to fully open (aslimited by the hinge). The varying degrees of openness may be defined asthe “gripper”.

In summary therefore, the configuration variables provided by thekinematic block 118 codes are as follows:

-   -   q_(ins): represents a distance from the slave base position 128        defined by axes x_(s), y_(s), and z_(s) to the first position        330 defined by axes x₃, y₃ and z₃ where the s-segment 130 of the        tool positioning device 81 begins;    -   δ_(prox): represents a first bend plane in which the s-segment        130 is bent relative to the x_(s)-y_(s) plane of the fixed slave        reference frame;    -   θ_(prox): represents an angle at which the first and second        sections 320 and 322 of the s-segment 130 is bent in the first        bend plane;    -   θ_(dist): represents a second bend plane in which the distal        segment 132 is bent relative to the x_(s)-y_(s) plane of the        fixed slave reference frame;    -   θ_(dist): represents an angle through which the distal segment        132 is bent in the second bend;    -   γ: represents a rotation of the end effector 73 about axis z₂;        and    -   Gripper: represents a degree of openness of the gripper jaws of        the end effector 73. (This is a value which is calculated in        direct proportion to a signal produced by an actuator (not        shown) on the handle 102 indicative of an amount of pressure the        operator exerts by squeezing the handle).

To calculate the configuration variables, it will first be recalled thatthe end effector rotation matrix R_(EENEW) is a 3×3 matrix:

$\begin{bmatrix}x_{2x} & y_{2x} & z_{2x} \\x_{2y} & y_{2y} & z_{2y} \\x_{2z} & y_{2z} & z_{2z}\end{bmatrix}.$Since the last column of R_(EENEW) is the z-axis of the end effectorreference frame written relative to the fixed slave reference framex_(s), y_(s) and z_(s), the values θ_(dist), θ_(dist), and γ associatedwith the distal segment 132 can be calculated according to therelations:

$\begin{matrix}{\theta_{dist} = {\frac{\pi}{2} - {{atan}\; 2\left( {\sqrt{z_{2x}^{2} + z_{2y}^{2}},\ z_{2z}} \right)}}} & (2) \\{\delta_{dist} = {{- {atan}}\; 2\left( {z_{2y},z_{2x}} \right)}} & (3)\end{matrix}$

${{If}\mspace{14mu}{\delta_{dist}}} > \frac{\pi}{2}$γ=a tan 2(−y _(2z) ,y _(2z))−δ_(dist)+π  (4a)elseγ=a tan 2(y _(2z) ,−x _(2z))−δ_(dist)  (4b)

These values can then be used to compute the locations of the thirdposition 334, the fourth position 336, and the end effector position 150relative to the fixed slave base position 128. The locations may beexpressed in terms of vectors p _(3/s) from the fixed slave baseposition 128 to the first position 330, p _(4/3) from the third position334 to the fourth position 336, and p _(5/4) from the fourth position336 to the end effector position 150. p _(3/s) is then calculated from{right arrow over (p)}_(EENEW) as follows:p _(3/s)= p _(EENEW)− p _(4/3)− p _(5/4),  (5)where:

$\begin{matrix}{{{\overset{¯}{p}}_{4/3} \cdot \overset{\_}{i}} = \frac{{- L_{2}}\cos{\delta_{dist}\left( {{\sin\theta_{dist}} - 1} \right)}}{\frac{\pi}{2} - \theta_{dist}}} & \left( {6a} \right) \\{{{\overset{\_}{p}}_{4/3} \cdot \overset{\_}{j}} = \frac{L_{2}\sin{\delta_{dist}\left( {{\sin\theta_{dist}} - 1} \right)}}{\frac{\pi}{2} - \theta_{dist}}} & \left( {6b} \right) \\{{{\overset{¯}{p}}_{4/3} \cdot \overset{\_}{k}} = \frac{L_{2}{\cos\left( \theta_{dist} \right)}}{\frac{\pi}{2} - \theta_{dist}}} & \left( {6c} \right) \\{{{\overset{\_}{p}}_{5/4} \cdot \overset{\_}{i}} = {L_{3}{\cos\left( \delta_{dist} \right)}{\cos\left( \theta_{dist} \right)}}} & \left( {7a} \right) \\{{{\overset{\_}{p}}_{5/4} \cdot \overset{\_}{j}} = {{- L_{3}}{\sin\left( \delta_{dist} \right)}{\cos\left( \theta_{dist} \right)}}} & \left( {7b} \right) \\{{{{\overset{\_}{p}}_{5/4} \cdot \overset{\_}{k}} = {L_{3}{\sin\left( \theta_{dist} \right)}}},} & \left( {7c} \right)\end{matrix}$where:

ī is a unit vector in the x direction;

j is a unit vector in the y direction; and

k is a unit vector in the z direction.

Once the vector from the fixed slave base position 128 to the thirdposition 334 (p _(3/s)) is known, the configuration variables, δ_(prox)and θ_(prox), for the s-segment 130 can be found. The configurationvariable δ_(prox) associated with the s-segment 130 is calculated bysolving the following two equations for δ_(prox):

$\begin{matrix}{{{\overset{¯}{p}}_{3/s} \cdot \overset{\_}{i}} = \frac{{- L_{1}}\cos{\delta_{prox}\left( {{\sin\theta_{prox}} - 1} \right)}}{\frac{\pi}{2} - \theta_{prox}}} & \left( {8a} \right) \\{{{\overset{¯}{p}}_{3/s} \cdot \overset{\_}{j}} = {\frac{L_{1}\sin{\delta_{prox}\left( {{\sin\theta_{prox}} - 1} \right)}}{\frac{\pi}{2} - \theta_{prox}}.}} & \left( {8b} \right)\end{matrix}$The ratio of (8b) and (8a) givesδ_(prox)=a tan 2(− p _(3/s)· j , p _(3/s) ·i)  (9)

where ī and j are unit vectors in the x and y directions respectively.

A closed form solution cannot be found for θ_(prox), thus θ_(prox) mustbe found with a numerical equation solution to either of equations (8a)or (8b). A Newton-Raphson method, being a method for iterativelyapproximating successively better roots of a real-valued function, maybe employed, for example. The Newton-Raphson method can be implementedusing the following equations:

$\begin{matrix}{{{f\left( \theta_{prox} \right)} = {{{\frac{L_{1}}{\frac{\pi}{2} - \theta_{prox}}\cos{\delta_{prox}\left( {1 - {\sin\;\theta_{prox}}} \right)}} - {{\overset{¯}{p}}_{3/s} \cdot \overset{\_}{i}}} = 0}},} & (10)\end{matrix}$where ī is the unit vector in the x direction.

The equation (10) is equation (8a) rewritten in the form f(θ_(prox))=0.The Newton-Raphson method tends to converge very quickly because in therange 0<θ_(prox)<π, the function has a large radius of curvature and hasno local stationary points. Following the Newton-Raphson method,successive improved estimates of θ_(prox) can be made iteratively tosatisfy equation (10) using the following relationship:

$\begin{matrix}{\theta_{n + 1} = {\theta_{n} - \frac{f\left( \theta_{n} \right)}{f^{\prime}\left( \theta_{n} \right)}}} & (11)\end{matrix}$Finally, upon determination of θ_(prox), the following equation can beused to find q_(ins),

$\begin{matrix}{{q_{ins} = {{{- {\overset{¯}{p}}_{3/s}} \cdot \overset{\_}{k}} - \frac{L_{1}\cos\theta_{prox}}{\frac{\pi}{2} - \theta_{prox}}}},} & (12)\end{matrix}$where:

k is the unit vector in the z direction;

p _(3/s)·k is the dot product of the vector p _(3/s) and the unit vectork.

The codes in the kinematics block 118 shown in FIG. 6 direct the masterapparatus 64 to calculate values for the above configuration variablesin response to the end effector position and orientation signals {rightarrow over (P)}_(EENEW) and R_(EENEW) produced by the end effectorposition and orientation calculation block 116 and these calculatedconfiguration variables generally define a tool positioning device poserequired to position the end effector 73 at a desired location and at adesired orientation in the end effector workspace.

It will be appreciated that configuration variables are produced foreach end effector 71 and 73 and therefore in the embodiment shown, twosets of configuration variables which will be referred to as left andright configuration variables respectively are produced and forwarded orotherwise made available to the motion control block 120.

Referring to FIG. 5, the master apparatus 64 queries the control unit 92for the current handle position vector {right arrow over (P)}_(MCURR)and current handle rotation matrix R_(MCURR) periodically, at a samplerate of about 1 kHz. These values are stored by the master apparatus 64in a first “current” buffer 140 having a first store 142 storing thethree values representing the currently acquired handle position vector{right arrow over (P)}_(MCURR) and a second store 144 storing the ninevalues representing the acquired handle rotation matrix R_(MCURR).

Referring to FIGS. 2 and 5, the master apparatus 64 also stores valuesx_(mb), y_(mb), z_(mb) representing a definable master base positionrepresented by a base position vector

_(MBASE) in a third store 146 and stores values representing a definablemaster base rotation matrix R_(MBASE) in a fourth store 148. The masterapparatus 64 initially causes the definable master base position vector

_(MBASE) to be set equal to the current handle position vector

_(MCURR) on startup of the system and causes the definable master baserotation matrix R_(MBASE) to define an orientation that is the same asthe orientation defined by the current handle rotation matrix R_(MCURR)associated with the current handle rotation, on startup of the system.

Initially, therefore:

-   -   _(MBASE)=        _(MCURR); and    -   R_(MBASE)=R_(MCURR).

In other words, the master base reference frame and the handle referenceframe coincide at startup.

Thereafter, the master base position vector

_(MBASE) and the master base rotation matrix R_(MBASE) are maintained atthe same values as on startup until the enablement signal is activated,such as by inactivation of the footswitch (170 in FIGS. 1 and 3), whichcauses the enablement signal to transition from the inactive state tothe active state. In response, the base setting block 216 shown in FIG.3 and more in detail in FIG. 6 is executed to change the master baseposition vector

_(MBASE) and master base rotation matrix R_(MBASE) to the currentlyacquired handle position vector

_(MCURR) and currently acquired handle orientation matrix R_(MCURR)respectively.

Referring to FIGS. 5 and 7, the master apparatus 64 further storesvalues x_(sb), y_(sb), z_(sb) representing a definable slave baseposition vector

_(EEBASE) in a fifth store 152 and stores values representing adefinable slave base rotation matrix R_(EEBASE) in a sixth store 154.The master apparatus 64 initially causes the definable slave baseposition vector

_(EEBASE) to be set equal to the new end effector position vector

_(EENEW) and causes the definable slave base rotation matrix R_(EEBASE)to define an orientation that is the same as the orientation defined bythe new end effector rotation matrix R_(EENEW) on startup of the system.

Initially, therefore:

-   -   _(EEBASE)=        _(EENEW); and    -   R_(EEBASE)=R_(EENEW).

In other words, the slave base reference frame and the end effectorreference frame coincide at startup.

The slave base position

_(EEBASE) and slave base rotation matrix R_(EEBASE) are maintained atthe same values as on startup until the enablement signal is activatedsuch as by inactivation of the footswitch (170 in FIGS. 1 and 3), whichcauses the enablement signal to transition from the inactive state tothe active state. In response, the base setting block 216 in FIG. 6directs the master apparatus 64 to change the slave base position vector

_(EEBASE) and slave base rotation matrix R_(EEBASE) to the newlycalculated end effector position vector

_(EENEW) and newly calculated end effector rotation matrix R_(EENEW).

Referring to FIGS. 3 and 8A to 8D, the end effector position andorientation calculation block 116 is executed each time a set of newvalues for

_(MCURR) and R_(MCURR) are acquired from the control unit 92. It beginswith a first block 161 that directs the master apparatus 64 to check thestate of the enablement signal to determine whether it is active orinactive. If the enablement signal is active, then the process continuesat block 160 shown in FIG. 8B. Block 160 directs the master apparatus 64to produce and store, in a seventh store 162 in FIG. 5, valuesrepresenting the new end effector position vector

_(EENEW) and to produce and store, in an eighth store 164 in FIG. 5,values representing the new end effector rotation matrix R_(EENEW).

To produce new end effector position signals

_(EENEW) and new end effector orientation signals R_(EENEW) representinga desired end effector position 150 and desired end effectororientation, relative to the slave base position 128 and the slave baserotation, the new end effector position signals

_(EENEW) and new end effector orientation signals R_(EENEW) arecalculated according to the following relations:

_(EENEW) =A(

_(MCURR)−

_(MBASE))+

_(EEBASE)andR _(EENEW) =R _(EEBASE) R _(MBASE) ⁻¹ R _(MCURR)

-   Where:    _(EENEW) is the new end effector position vector that represents the    new desired end effector position 150 of the end effector 73 in the    end effector workspace, relative to the slave base reference frame;    -   A is a scalar value representing a scaling factor in        translational motion between the master and the slave;    -   _(MCURR) is the current representation of the handle position        vector stored in the first store 142, the handle position vector        being relative to the fixed master reference frame;    -   _(MBASE) is the last-saved position vector        _(MCURR) for the handle 102 that was saved upon the last        inactive to active state transition of the enablement signal        such as by release of the footswitch 170 or on system        initialization or by operation of a control interface by the        operator;    -   _(EEBASE) is the last-saved position vector        _(EENEW) for the end effector 73 that was saved upon the last        inactive to active state transition of the enablement signal;    -   R_(EENEW) is the new end effector rotation matrix representing        the current orientation of the end effector 73 relative to the        slave reference frame;    -   R_(EEBASE) is the rotation matrix representing the last-saved        orientation of the end effector 73 saved upon the last inactive        to active state transition of the enablement signal;    -   R_(MBASE) ⁻¹ is the inverse of rotation matrix R_(MBASE), where        R_(MBASE) is a rotation matrix representing the last-saved        orientation of the handle 102 saved upon the last inactive to        active state transition of the enablement signal;    -   R_(MCURR) is the currently acquired rotation matrix representing        the orientation of the handle 102 relative to the fixed master        reference frame.

The following describes how the master apparatus 64 is controlled by thecodes in the end effector position and orientation calculation block 116to effect autonomous alignment of the orientation of the end effector 73with the handle 102 after clutching and to effect autonomous alignmentof the z-axes of the handle 102 and end effector 73 for wrist rollmanagement.

Referring to FIG. 8B, after values representing the desired end effectorposition and orientation signals

_(EENEW) and R_(EENEW) are calculated by the master apparatus 64executing block 160, the master apparatus 64 is directed to block 202which directs the master apparatus to determine whether the system 50 isconfigured to allow wrist roll management. A simple binary wrist rollmanagement signal selectively set by the operator is used to indicate tothe master apparatus 64 whether the system 50 is configured to allowwrist roll management or not. If the system is not configured for wristroll management, block 204 directs the master apparatus 64 to compute arotation matrix that carries the newly calculated end effectororientation into the current handle orientation (R_(EE_TO_MASTER)) bythe relation:R _(EE_TO_MASTER) =R _(EENEW) ⁻¹ R _(MCURR)

-   Where: R_(EENEW) ⁻¹ is the inverse matrix of the end effector    rotation matrix R_(EENEW) represented by a 3×3 matrix stored in the    eighth store 164 in FIG. 3; and    -   R_(MCURR) is the current handle rotation matrix represented by        the 3×3 matrix stored in the second store 144 in FIG. 3.

Then, block 206 directs the master apparatus 64 to compute an angle ofrotation associated With R_(EE_TO_MASTER) (Φ_(EE_TO_MASTER)) by therelation:Φ_(EE_TO_MASTER)=a cos (0.5 trace(R _(EE_TO_MASTER))−1)

This angle of rotation (Φ_(EE_TO_MASTER)) represents the alignmentdifference between the orientation of the handle 102 and the newlycalculated end effector orientation.

In a special case, applicable to the embodiment described here, it isdesirable that to be aligned, only the z-axes of the reference framesdescribed by R_(EENEW) and R_(MCURR) be coincident. In this case thehandle 102 and the end effector 73 point in the same direction relativeto their respective fixed reference frames (x_(r), y_(r), z_(r) andx_(s), v_(s), z_(s), respectively) and the roll about their z-axis isnot considered.

In this special case therefore, blocks 204 and 206 shown in FIG. 8B arereplaced with block 205 shown in FIG. 9 which involves the followingcomputationΦ_(EE_TO_MASTER)=a cos(R _(EENEW)(1,3)*R _(M_CURR)(1,3)+R_(EENEW)(2,3)*R _(M_CURR)(2,3)+R _(EENEW)(3,3)*R _(MCURR)(3,3))

-   -   Where (i,j) represent matrix row (i) and column (j) indices.

This computation represents the angle obtained from the dot product ofthe z-axes of the handle reference frame and end effector referenceframe.

Referring back to FIG. 8B, alternatively, if at block 202 it isdetermined that the system is configured to allow wrist roll management,block 208 directs the master apparatus 64 to compute an offset anglebetween the handle reference frame and the end effector reference framez-axes (Φ_(EE_TO_MASTER)) by the relation:

$\phi_{EE_{-}TO_{-}MASTER} = {{acos}\left( {\begin{Bmatrix}R_{{EENEW1},3} \\R_{{EENEW2},3} \\R_{{EENEW3},3}\end{Bmatrix} \cdot \begin{Bmatrix}R_{{M{CURR}\; 1},3} \\R_{{MCURR2},3} \\R_{{M{CURR}\; 3},3}\end{Bmatrix}} \right)}$

After executing either block 206 or block 208, the angle of rotation bywhich the handle 102 and end effector 73 are out of alignment, i.e. thealignment error, is given by Φ_(EE_TO_MASTER).

The master apparatus 64 is then directed to block 210 which causes it todetermine whether the alignment error Φ_(EE_TO_MASTER) meets acriterion, such as being above a threshold value. If the alignment erroris not above the threshold value, the current handle orientationR_(MCURR) and new end effector orientation R_(EENEW) are considered tobe aligned.

Then, block 215 directs the master apparatus 64 to signal the motioncontrol block 120 of FIG. 3 to indicate that slave control signals basedon the newly calculated values for

_(EENEW) and R_(EENEW) are to be sent to the slave computer 74. Thiscauses the end effector 73 to assume a position and orientationdetermined by the current position and current orientation of the handle102 when the alignment difference meets the criterion.

Block 159 then directs the master apparatus 64 to copy the newlycalculated end effector position vector

_(EENEW) and end effector rotation matrix R_(EENEW) into an eleventh andtwelfth stores 147 and 149 of a previous buffer 141 of FIG. 5. The newlycalculated end effector position vector

_(EENEW) and newly calculated end effector rotation matrix R_(EENEW) arethus renamed as “previously calculated end effector position vector”

_(EEPREV) and “previously calculated end effector rotation matrix”R_(EEPREV). By storing the newly calculated end effector position vector

_(EENEW a)nd newly calculated end effector rotation matrix R_(EENEW), aspreviously calculated end effector position vector

_(EEPREV) and previously calculated end effector rotation matrixR_(EEPREV), a subsequently acquired new end effector position vector

_(EENEW) and subsequently acquired new end effector rotation matrixR_(EENEW) can be calculated from the next current handle position vector

_(MCURR) and next current handle position matrix R_(MCURR).

The end effector position and orientation calculation block 116 is thuscompleted, and the calculated

_(EENEW) and R_(EENEW) values stored in the seventh and eighth stores162 and 164 are available for use by the kinematics block 118.

If at block 210 the alignment error is above the threshold value, block214 directs the master apparatus 64 to produce a rotation matrix thatcarries the previous handle orientation into current handle orientation(R_(DIFF)), according to the relation:R _(DIFF) =R _(MPREV) ⁻¹ R _(MCURR)

-   -   Where: R_(MPREV) ⁻¹ is the inverse of the previous handle        rotation matrix stored in the tenth store 145 of FIG. 3; and        -   R_(MCURR) is the current handle rotation matrix stored in            the second store 144 of FIG. 3.

Referring to FIG. 8C, the master apparatus 64 is then directed by block230 to determine a unit vector in the direction of an axis of rotation(e_(DIFF)) associated with R_(DIFF) by the relation:

${\overset{¯}{p} = {{0.5}\begin{Bmatrix}{R_{{{DIFF}\; 3},2} - R_{{{DIFF}\; 2},3}} \\{R_{{{DIFF}\; 1},3} - R_{{{DIFF}\; 3},1}} \\{R_{{{DIFF}\; 2},1} - R_{{{DIFF}\; 1},2}}\end{Bmatrix}}}{{\overset{¯}{e}}_{DIFF} = \frac{\overset{¯}{p}}{\overset{\_}{p}}}$

Then, block 216 directs the master computer 64 to compute an angle ofrotation (Φ_(DIFF)) associated with R_(DIFF) by the relation:Φ_(DIFF)=a cos (0.5 trace(R _(DIFF))−1)

Then, block 218 directs the master computer 64 to compute an angularspeed of the rotation (ω_(DIFF)) associated with R_(DIFF) by therelation:ω_(DIFF) =sr·Φ _(DIFF)

-   -   Where: sr=sample rate in Hz at which {right arrow over        (P)}_(MCURR) and R_(MCURR) values are acquired from the control        unit (92).

Then, block 220 directs the master apparatus 64 to determine whether theangular speed WDIFF meets a second criterion such as being above athreshold speed to initiate auto alignment. (This may avoid anyautomated motion when the user is performing slow fine movements thatmay be undesirable.)

If not, the master apparatus 64 is directed to block 222 whichcorresponds to location “E” on FIG. 8B to execute blocks 215 and 159 andthen exit the end effector position and orientation calculation block116. The saved 15 {right arrow over (P)}_(EENEW) and R_(EENEW) values inthe seventh and eighth stores 162 and 164 of FIG. 3 are then availablefor use by the kinematics block 118.

If at block 220 the angular speed ω_(DIFF) is above the threshold speed,block 224 directs the master computer 64 to determine whether the systemis configured to allow wrist roll misalignment by reading the status ofthe wrist roll management signal set by the operator. If the wrist rollmanagement signal is not active, block 226 directs the master apparatus64 to determine a misalignment axis e_(ERR), i.e. the axis of rotationassociated with R_(EE_TO_MASTER) by the relation

$\overset{¯}{q} = {{0.5}\begin{Bmatrix}{R_{{{EE}\;\_\;{TO}\;\_\;{MASTER}\; 3},2} - R_{{{EE}\;\_\;{TO}\;\_\;{MASTER}\; 2},3}} \\{R_{{{EE}\;\_\;{TO}\;\_\;{MASTER}\; 1},3} - R_{{{EE}\;\_\;{TO}\;\_\;{MASTER}\; 3},1}} \\{R_{{{EE}\;\_\;{TO}\;\_\;{MASTER}\; 2},1} - R_{{{EE}\;\_\;{TO}\;\_\;{MASTER}\; 1},2}}\end{Bmatrix}}$${\overset{¯}{e}}_{ERR} = \frac{\overset{¯}{q}}{\overset{\_}{q}}$

Alternatively, if the system is configured to allow wrist rollmisalignment, block 228 directs the master apparatus 64 to determine aunit vector in the direction of the misalignment axis e_(ERR) by therelation:

$e_{ERR} = {\frac{1}{\sin\left( \phi_{{EE}\;\_\;{TO}\;\_\;{MASTER}} \right)}\begin{Bmatrix}R_{{{EENEW}\; 1},3} \\R_{{E{ENEW}\; 2},3} \\R_{{E{ENEW}\; 3},3}\end{Bmatrix} \times \begin{Bmatrix}R_{{{MCURR}\; 1},3} \\R_{{{MCURR}\; 2},3} \\R_{{MCURR3}\;,3}\end{Bmatrix}}$

Now referring to FIG. 8D, block 232 directs the master apparatus 64 tocompute the component of the handle rotation in the misalignment plane Lby the relation:L=| e _(DIFF)· e _(ERR)|

Then, block 234 directs the master apparatus 64 to compute a correctionangle Φ_(C), as a function of the angle of the master rotation Φ_(DIFF)and the component of the master rotation in the misalignment plane L bythe relation:Φ_(C) =BΦ _(DIFF) f(L)

-   Where: B is a scaling factor that defines the fraction of Φ_(DIFF)    that should be used in the correction angle Φ_(C), for example    B=0.5.    -   f(L) is a function that alters the profile of how the correction        angle Φ_(C) changes with the plane of orientation change, for        example f (L)=L³.

Then, block 236 directs the master apparatus 64 to compute a correctionmatrix R_(C)(Φ_(C), e_(ERR)) to rotate by the correction angle Φ_(C)about the misalignment axis e_(ERR).

The correction matrix R_(C) is determined by the relation:R _(C)(Φ_(C) ,e _(ERR))=e _(ERR) e _(ERR) ^(T)+COS (Φ_(C))*(I-e _(ERR) e_(ERR) ^(T))+sin (Φ_(C))E _(ERR)Where:

E_(ERR) = the  cross  product  matrix  of  vector  e_(ERR)$E_{ERR} = \begin{bmatrix}0 & {- e_{ERR3}} & e_{ERR2} \\e_{ERR3} & 0 & {- e_{ERR1}} \\{- e_{ERR2}} & e_{ERR1} & 0\end{bmatrix}$$I = {{the}\mspace{14mu} 3 \times 3\mspace{14mu}{identity}\mspace{14mu}{{matrix}\mspace{14mu}\begin{bmatrix}1 & 0 & 0 \\0 & 1 & 0 \\0 & 0 & 1\end{bmatrix}}}$

Then, block 238 directs the master apparatus 64 to modify the endeffector base matrix using the correction matrix, R_(C) by the relation:R _(EEBASE) ′=R _(EEBASE) R _(C)

The R_(EEBASE)′ value calculated by block 238 is then saved in the sixthstore 154, of FIG. 3 as R_(EEBASE).

Then, block 244 directs the master apparatus 64 to re-compute the endeffector rotation matrix R_(EENEW) using the new end effector baserotation matrix R_(EEBASE) and store this new end effector orientationas R_(EENEW) in store 164, of FIG. 3.R _(EENEW) =RR _(EEBASE)

-   -   Where: R is a rotation matrix describing the rotation between        the master base rotation matrix and R_(MBASE) the current handle        rotation matrix R_(MCURR).        R=R _(MCURR) R _(MBASE) ⁻¹

Then, block 240 directs the master apparatus 64 to location “E” in FIG.8B to execute blocks 215 and 159 and the end effector positioning andorientation calculation block 116 is ended.

Alternatively referring to FIG. 8E blocks 250 and 252 can replace blocks238 and 244 in FIG. 8D and the use of base frames R_(MBASE) andR_(EEBASE) to calculate R_(MCURR) and R_(EENEW) can be avoided,requiring consideration of only the reference frames x_(r), y_(r), z_(r)and x_(s), y_(s), z_(s) for the master and slave, respectively, and thetask frames x₁, y₁, z₁ and x₂, y₂, z₂ for the master and slave,respectively. In this alternative, the misalignment R_(Δ) between R_(m)and R_(ee) is calculated upon system activation using the relationR_(Δ)=R_(m) ⁻¹R_(ee). The master-slave misalignment (R_(Δ)) is then usedto account for the misalignment as the new values of R_(ee) are computedusing the relation R_(EENEW)=R_(MCURRR)R_(Δ). A correction matrix(R_(C)) is then used in block 250 to adjust R_(Δ) each time step usingthe relation R_(Δ)′=R_(Δ)R_(C) until the alignment reaches the alignmentcriterion.

Referring back to FIG. 3, the kinematics block 118 then directs themaster apparatus 64 to use the saved values for

_(EENEW) and

_(EENEW) to determine the configuration variables. The configurationvariables are made available to the motion control block 120 and themotion control block 120 produces the slave control signals representingwire lengths. Referring back to FIG. 8A, block 161 has the effect thatthe slave control signals based on

_(EENEW) and R_(EENEW) are transmitted to the slave computer 74 when thefootswitch 170 is not depressed. When the footswitch 170 is depressedthe process continues at block 163, and the slave control signals basedon

_(EEPREV) and R_(EEPREV) are transmitted to the slave computer 74 whenthe footswitch 170 is depressed.

It will be appreciated that the above routine is executed by the endeffector position and orientation calculation block 116 after eachsample of the {right arrow over (P)}_(MCURR) and R_(MCURR) values isacquired from the control unit 92. Suitable values for the scalingfactor B at block 234 and suitable choices for the correction anglefunction f(L) at block 234 will cause the R_(EEBASE) saved in store 154to be updated each time the routine is executed, i.e. each time new{right arrow over (P)}_(MCURR) and R_(MCURR) values are acquired, untilthe alignment error Φ_(EE_TO_MASTER) is below the alignment threshold atwhich time the handle 102 and the end effector 73 are considered to bealigned and no further modification of the R_(EEBASE) value occurs.

Generally, when the enablement signal is in the inactive state, thehandle 102 on input device 61 can be moved and rotated and thecalculations of {right arrow over (P)}_(EENEW) and R_(EENEW) will stillbe performed, but there will be no movement of the end effector 73. Thisallows “clutching” or repositioning the handle 102 without correspondingmovement of the end effector 73, to enable the end effector 73 to haveincreased range of movement and to allow the operator to repositiontheir hands to a more comfortable position within the mastertranslational workspace. For example, referring to FIG. 7, the operatorcould push the handle 102 in the z_(r) direction toward the control unit92 while the enablement signal is active, whereby the end effector 73 ismoved in the z_(s) direction corresponding to the movement of the handle102. Then, the operator can actuate the footswitch 170 to set theenablement signal inactive and while the footswitch is actuated,withdraw the handle 102 along the z_(r) axis in the opposite direction,away from the control unit 92, while the end effector 73 remainsstationary due to the enablement signal being inactive. Then, theoperator can release the footswitch 170 to set the enablement signalactive and then continue pushing the handle 102 in the z_(r) directiontoward the control unit 92 while the end effector 73 is moved in thez_(s) direction corresponding to the movement of the handle 102. Similareffects are experienced with rotational movements of the handle 102 andthe end effector 73, when the enablement signal is active and inactive,to achieve a clutching effect in rotation.

The above clutching effect is achieved by causing movements of thehandle 102 and movements of the end effector 73 to be made relative tothe last-saved master base position

_(MBASE) and orientation R_(MBASE) and the last saved slave baseposition

_(EEBASE) and orientation R_(EEBASE) respectively. The master computer64 stores the current values of the current handle position

_(MCURR) and current handle orientation R_(MCURR) signals as new valuesof the master base position signals

_(MBASE) and new values of the master base orientation signals R_(MBASE)respectively, and stores the current values of the end effector positionsignals

_(EENEW) and new end effector orientation R_(EENEW) signals as newvalues of the slave base position signals

_(EEBASE) and new values of the slave base orientation signalsR_(EEBASE) respectively, in response to the enablement signaltransitioning from the “not active” state to the “active” state.Otherwise, upon release of the footswitch 170, the end effector 73 would“snap” to the absolute position directly mapped to the position andorientation of the handle 102 and this could be dangerous if it were tooccur inside a patient because the end effector 73 could tear intotissue or internal organs of the patient with possibly life-threateningeffects. In addition, the surgeon would feel somewhat out of control ofthe end effectors 71 and 73.

While the above described clutching effect is desirable to match therange of translational movement of the end effector 73 with the range ofmovement of the handle 102 and to reposition the hands of the operatorto a comfortable position, it is not desirable for clutching to resultin reorientation of the handle within the master rotational workspacebecause orientation control can become unnatural or unintuitive to theoperator when there is a misalignment between the handle 102 and endeffector 73. In the absence of a mechanical means to maintain theorientation of the handle 102 it would be difficult for the operator torotate the handle 102 to cause it to be exactly aligned with the endeffector 73 on release of the footswitch 170 so that normal operationcan be resumed. In this regard, the codes of the end effector positionand orientation calculation block 116 direct the master apparatus 64 todetect a difference, between the current handle orientation signalsR_(MCURR) and the new end effector orientation signals R_(EENEW), thedifference representing a difference in physical alignment between theend effector 73 and the handle 102 relative to their respective fixedreference frame. In response to detecting the difference, the codescause the master apparatus 64 to adjust the saved slave base orientationsignals R_(EEBASE) to ultimately have values close to the same values asthe saved master base orientation signals R_(MBASE) so that subsequentgenerations of the end effector orientation signals R_(EENEW) cause theslave control signals produced by the motion control block 120 to causethe end effector 73 to be physically aligned with the handle 102relative to their respective fixed reference frames.

This technique of adjusting the saved slave base orientation signalsR_(EEBASE) also has applications in providing a wrist-roll managementfeature, where wrist roll is measured as variations of orientation ofthe handle 102 relative to only the z-axis. The wrist roll managementfeature would have the effect of correcting only for misalignment forthe direction in which the end effector 73 and the handle 102 arepointing and not the rotation about the axis 134 of the end effector 73.

Generally, the above described system may cause smooth autonomous motionof the end effector 73 toward alignment with the handle 102, when thereis a misalignment between the handle 102 and the end effector 73 withoutcompromising control of the end effector 73 for the operator. Inaddition, if the alignment error Φ_(EE_TO_MASTER) exceeds the thresholdvalue, the alignment error will always be reduced no matter whatdirection the handle 102 is moving, unless e_(DIFF) and e_(ERR) areparallel and f(A) is such that f(A)=0 when A=0.

While specific embodiments of the invention have been described andillustrated, such embodiments should be considered illustrative of theinvention only and not as limiting the invention as construed inaccordance with the accompanying claims.

What is claimed is:
 1. A method of reducing alignment difference betweenan instrument end effector and an input actuator in a robotic system,the method comprising, by a processor: receiving signals representing anorientation of the input actuator; receiving an enablement signal that,in an active state, permits a change in the orientation of theinstrument end effector in response to a change in the orientation ofthe input actuator; in response to determining that the enablementsignal has transitioned to the active state, determining an alignmenterror between a current orientation of the input actuator and a currentorientation of the instrument end effector; and while the alignmenterror is greater than an alignment criterion and the enablement signalis in the active state: determining a rotational speed associated with achange in orientation of the input actuator; in response to determiningthat the rotational speed satisfies a threshold rotational speedassociated with reducing the alignment error, producing an adjustmentcontrol signal; and adjusting the orientation of the instrument endeffector based on the adjustment control signal to reduce the alignmenterror.
 2. The method of claim 1 wherein determining the alignment errorcomprises computing an offset angle between a master reference frame inwhich the input actuator is oriented and a slave reference frame inwhich the instrument end effector is oriented.
 3. The method of claim 2wherein computing the offset angle is performed while permitting a rollalignment error about the respective z-axes between the slave referenceframe and the master reference frame.
 4. The method of claim 2 whereincomputing the offset angle comprises generating a rotation matrix thatcarries the instrument end effector orientation at a time of theenablement signal transition into the input actuator orientation at thetime of the enablement signal transition and determining the offsetangle associated with the rotation matrix.
 5. The method of claim 2wherein the alignment error is determined to be greater than thealignment criterion in response to the offset angle between the slavereference frame and the master reference frame being less than athreshold value.
 6. The method of claim 1 wherein producing theadjustment control signal comprises: determining a misalignment axisassociated with the alignment error; determining an axis and an angle ofrotation associated with the change in orientation of the inputactuator; computing a component of rotation associated with the changesin orientation of the input actuator in a direction of the misalignmentaxis; and computing a correction angle for the adjustment based on theangle of rotation of the input actuator and the component of rotation.7. The method of claim 1 further comprising, in response to determiningthat the rotational speed does not satisfy the threshold rotationalspeed, not adjusting the orientation of the instrument end effector. 8.The method of claim 1 wherein determining the rotational speedcomprises: generating a rotation matrix that carries a previous inputactuator orientation into a current input actuator orientation;computing an angle of rotation associated with the rotation matrix; andcomputing the rotational speed based on the angle of rotation and asampling rate for the input actuator.
 9. The method of claim 1 whereinproducing the adjustment control signal; comprises successivelyadjusting an orientation of a slave reference frame until the slavereference frame in which the instrument end effector is oriented meetsthe alignment criterion.
 10. The method of claim 1 wherein determiningthe rotational speed is performed for each change in orientation of theinput actuator.
 11. A non-transitory computer readable medium storinginstructions that, when executed by a processor, cause the processor toexecute a method of reducing alignment difference between an instrumentend effector and an input actuator in a robotic system, the methodcomprising: receiving signals representing an orientation of the inputactuator; receiving an enablement signal that, in an active state,permits a change in the orientation of the instrument end effector inresponse to a change in the orientation of the input actuator; inresponse to determining that the enablement signal has transitioned tothe active state, determining an alignment error between a currentorientation of the input actuator and a current orientation of theinstrument end effector; and while the alignment error is greater thanan alignment criterion and the enablement signal is in the active state:determining a rotational speed associated with a change in orientationof the input actuator; in response to determining that the rotationalspeed satisfies a threshold rotational speed associated with reducingthe alignment error, producing an adjustment control signal; andadjusting the orientation of the instrument end effector based on theadjustment control signal to reduce the alignment error.
 12. Thecomputer readable medium of claim 11 wherein determining the alignmenterror comprises computing an offset angle between a master referenceframe in which the input actuator is oriented and a slave referenceframe in which the instrument end effector is oriented.
 13. The computerreadable medium of claim 12 wherein computing the offset angle isperformed while permitting a roll alignment error about the respectivez-axes between the slave reference frame and the master reference frame.14. The computer readable medium of claim 12 wherein computing theoffset angle comprises generating a rotation matrix that carries theinstrument end effector orientation at a time of the enablement signaltransition into the input actuator orientation at the time of theenablement signal transition and determining the offset angle associatedwith the rotation matrix.
 15. The computer readable medium of claim 12wherein the alignment error is determined to be greater than thealignment criterion in response to the offset angle between the slavereference frame and the master reference frame being less than athreshold value.
 16. The computer readable medium of claim 11 whereinproducing the adjustment control signals comprises: determining amisalignment axis associated with the alignment error; determining anaxis and an angle of rotation associated with the change in orientationof the input actuator; computing a component of rotation associated withthe changes in orientation of the input actuator in a direction of themisalignment axis; and computing a correction angle for the adjustmentbased on the angle of rotation of the input actuator and the componentof rotation.
 17. The computer readable medium of claim 11 furthercomprising, in response to determining that the rotational speed doesnot satisfy the threshold rotational speed, not adjusting theorientation of the instrument end effector.
 18. The computer readablemedium of claim 11, wherein determining the rotational speed comprises:generating a rotation matrix that carries a previous input actuatororientation into a current input actuator orientation; computing anangle of rotation associated with the rotation matrix; and computing therotational speed based on the angle of rotation and a sampling rate forthe input actuator.
 19. The computer readable medium of claim 11 whereinproducing the adjustment control signals comprises successivelyadjusting an orientation of a slave reference frame until the slavereference frame in which the instrument end effector is oriented meetsthe alignment criterion.
 20. The computer readable medium of claim 11wherein determining the rotational speed is performed for each change inorientation of the input actuator.